Interaction Capture and Synthesis of Human Hands

This thesis addresses several issues in modelling interaction with human hands in computer graphics and animation. Modifying motion capture to satisfy the constraints of new animation is difficult when contact is involved because physical interaction involves energy or power transfer between the system of interest and the environment, and is a critical problem for computer animation of hands. Although contact force measurements provide a means of monitoring this transfer, motion capture as currently used for creating animation has largely ignored contact forces. We present a system of capturing synchronized motion and contact forces, called interaction capture. We transform interactions such as grasping into joint compliances and a nominal reference trajectory in an approach inspired by the equilibrium point hypothesis of human motor control. New interactions are synthesized through simulation of a quasi-static compliant articulated model in a dynamic environment that includes friction. This uses a novel position-based linear complementarity problem formulation that includes friction, breaking contact, and coupled compliance between contacts at different fingers. We present methods for reliable interaction capture, addressing calibration, force estimation, and synchronization. Additionally, although joint compliances are traditionally estimated with perturbation-based methods, we introduce a technique that instead produces estimates without perturbation. We validate our results with data from previous work and our own perturbation-based estimates. A complementary goal of this work is hand-based interaction in virtual environments. We present techniques for whole-hand interaction using the Tango, a novel sensor that performs interaction capture by measuring pressure images and accelerations. We approximate grasp hand-shapes from previously observed data through rotationally invariant comparison of pressure measurements. We also introduce methods involving heuristics and thresholds that make reliable drift-free navigation possible with the Tango. Lastly, rendering the skin deformations of articulated characters is a fundamental problem for computer animation of hands. We present a deformation model, called EigenSkin, which provides a means of rendering physicallyor example-based deformation models at interactive rates on graphics hardware.